feat: OTel integration in XRPLD#7770
Draft
pratikmankawde wants to merge 1502 commits into
Draft
Conversation
…se8-log-correlation
…ase9-metric-gap-fill
…hase2-rpc-tracing
…hase1c-rpc-integration
…XRPLF/rippled into pratik/otel-phase10-workload-validation
…cleaner) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…(misc-include-cleaner) std::ranges::transform/all_of come from <algorithm>, which is already included. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nclude-cleaner) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…hase1c-rpc-integration
…hase2-rpc-tracing
…-tx-tracing # Conflicts: # include/xrpl/telemetry/SpanGuard.h
…consensus-tracing
…phase5-docs-deployment
Signed-off-by: Pratik Mankawde <3397372+pratikmankawde@users.noreply.github.com>
Contributor
Author
|
/ai-review |
- OTelCollector::formatName made static (no member state); braces added; <cctype> added for std::tolower. - Telemetry.cpp getMeter(): braces around single-statement if. - GetMeter.cpp: drop unused <opentelemetry/metrics/meter.h> and sync_instruments.h; add <xrpl/beast/utility/Journal.h> and <memory>; make sdkProvider const. - ValidationTracker.h: add <cstddef> for std::size_t. - detail/ValidationTracker.cpp: use std::views::reverse range-based loop; add <ranges>. - test ValidationTracker.cpp: use auto for cast result. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…se8-log-correlation
…ase9-metric-gap-fill
…ase10-workload-validation
…ervice' to 'Service Name' The template variable's Prometheus label name is `service_name` but the UI display label read 'Service', which mismatched what users typed into Grafana's Metrics Drilldown UI (which defaults to filter on a bare `service` label). Rename the display label to 'Service Name' so it unambiguously points at the underlying `service_name` Prometheus label. Behaviour unchanged; template-variable name (`service_name`) and PromQL label are the same. Only the widget label shown at the top of the dashboard changes from 'Service' to 'Service Name'. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ervice' to 'Service Name' Continuation of the same rename applied on phase-6-statsd (commit 91c6582): 5 more dashboards introduced on phase-7 (formerly the system-* set, renamed to bare names by 2c590a4) still carried the older 'Service' UI label. Aligning them with the rest. Same reasoning as the phase-6 commit: the underlying Prometheus label is `service_name`, and the UI label 'Service' misled users into typing `service=xrpld` in Grafana Metrics Drilldown. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…se8-log-correlation
…hase1c-rpc-integration # Conflicts: # include/xrpl/telemetry/Telemetry.h
…hase2-rpc-tracing # Conflicts: # .github/scripts/levelization/results/loops.txt
…consensus-tracing
…phase5-docs-deployment # Conflicts: # include/xrpl/telemetry/Telemetry.h
…ve-metrics # Conflicts: # include/xrpl/telemetry/Telemetry.h # src/libxrpl/telemetry/TelemetryConfig.cpp
Convert leading /// blocks in these 9 phase3-originating files to house style /** */ (using the updated fix_doxy.py that now handles /// -> block conversion). Comment-only: code is byte-identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… refresh levelization Convert leading /// blocks to house-style /** */ across the telemetry files carried on this branch (using the updated fix_doxy.py). Also regenerate levelization results. Comment-only: code is byte-identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Convert leading /// blocks in LedgerSpanNames.h and PeerSpanNames.h to house-style /** */. Comment-only: code is byte-identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…consensus-tracing # Conflicts: # include/xrpl/telemetry/TraceContextPropagator.h # src/xrpld/telemetry/PropagationHelpers.h
…phase5-docs-deployment # Conflicts: # src/xrpld/telemetry/PropagationHelpers.h
…se8-log-correlation
xrpl_branch and xrpl_node_role are collector/infra-injected resource labels used by perf-iac dashboards to identify the build under test and its role in the perf cluster. They have no L1 span-attribute source or L6 native-metric-label source (like the existing job/instance/job_type builtins), so Rule D flagged them across every perf-iac dashboard. Register them in the builtins set, matching the existing pattern for infra-level dashboard labels. Add test coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ing Rule D" This reverts commit 48747ba.
…g Rule D xrpl_branch and xrpl_node_role are stamped by the perf-iac harness -- infrastructure outside this repo's OTel code -- so they have no L1 (*SpanNames.h), L2 (collector config), or L6 (MetricsRegistry.cpp) source Rule D can derive them from. Unlike the generic builtins set (Prometheus/ Grafana mechanics every OTel setup has), these are repo-specific and narrow, so they get their own EXTERNAL_INFRA_LABELS constant: a visible, documented, deliberately narrow exception to the 'no hardcoded allowlist' design principle, not a silent workaround. Add test coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…hase2-rpc-tracing
…consensus-tracing
…phase5-docs-deployment
…se8-log-correlation
…ase9-metric-gap-fill # Conflicts: # src/xrpld/telemetry/ValidationTracker.h
…ase9-metric-gap-fill
…ase10-workload-validation # Conflicts: # src/xrpld/telemetry/MetricsRegistry.h # src/xrpld/telemetry/ValidationTracker.h
…h comments An earlier Doxygen-style normalization pass on this file introduced a stray xrpld_ prefix into 5 comment references to the validation counters/gauge. The actual instrument names registered in MetricsRegistry.cpp have no such prefix (validation_agreements_total, validation_missed_total, validation_agreement). Comment-only fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ase10-workload-validation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
High Level Overview of Change
Adds full OpenTelemetry (OTel) integration to
xrpld: distributed traces, native OTel metrics, and log/trace correlation, with a companion observability stack (Collector, Tempo, Prometheus, Loki, Grafana) and a workload harness that validates the end-to-end signal path.Grafana Dashboards
Scope:
traceparentpropagation across RPC and peer boundaries.xrpldlogs so Loki entries deep-link into Tempo.Context of Change
xrpldhad only StatsD counters and free-form logs — no distributed traces, no structured attributes, and no way to correlate a slow RPC to the consensus round it landed in. This change introduces an OTel-native pipeline so operators can trace requests end-to-end, alert on structured SLIs, and jump from a log line to its trace.Delivered in ten internal phases (plan docs under
OpenTelemetryPlan/), landed here as a single feature branch offdevelop. Telemetry is off by default; enable via[telemetry]/[insight] server=otelin the config.API Impact
libxrplchange — adds telemetry primitives (SpanGuard,SpanNames, metrics registry) usable bylibxrplconsumers; existing symbols unchanged.traceparentrides existing peer message headers and is optional.No wire-format or JSON-RPC schema changes. Config gains an opt-in
[telemetry]stanza and an[insight] server=otelvalue.Test Plan
docker/telemetry/workload compose brings up 5 validators + full OTel stack;make telemetry-workload-upruns the RPC/TX load generators.